Skip to content

[emhancement](compaction) disable compaction pause on high memory by default#61288

Open
luwei16 wants to merge 1 commit intoapache:masterfrom
luwei16:codex/disable-compaction-pause-on-high-memory-default
Open

[emhancement](compaction) disable compaction pause on high memory by default#61288
luwei16 wants to merge 1 commit intoapache:masterfrom
luwei16:codex/disable-compaction-pause-on-high-memory-default

Conversation

@luwei16
Copy link
Contributor

@luwei16 luwei16 commented Mar 12, 2026

High compaction score keeps more rowset metadata,
delete bitmaps, and version graph state in
memory.

Pausing compaction on high memory makes those
high-score tablets harder to merge, so
metadata backlog and memory pressure become
harder to bring down.

Disable this pause by default to avoid
reinforcing that feedback loop.

…default

High compaction score keeps more rowset metadata,
delete bitmaps, and version graph state in
memory.

Pausing compaction on high memory makes those
high-score tablets harder to merge, so
metadata backlog and memory pressure become
harder to bring down.

Disable this pause by default to avoid
reinforcing that feedback loop.
@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@luwei16
Copy link
Contributor Author

luwei16 commented Mar 12, 2026

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 27749 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 85d56d3502b6df0512dc95e2173b421b5c6b05df, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17627	4531	4304	4304
q2	q3	10646	767	529	529
q4	4687	369	254	254
q5	7731	1232	1013	1013
q6	183	176	146	146
q7	808	855	656	656
q8	10233	1505	1378	1378
q9	5944	4773	4761	4761
q10	6340	1923	1650	1650
q11	472	269	248	248
q12	749	578	467	467
q13	18080	2948	2183	2183
q14	236	233	218	218
q15	945	819	811	811
q16	763	732	694	694
q17	709	845	437	437
q18	5996	5508	5272	5272
q19	1388	1008	629	629
q20	501	487	386	386
q21	4760	1938	1454	1454
q22	391	312	259	259
Total cold run time: 99189 ms
Total hot run time: 27749 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4723	4635	4718	4635
q2	q3	3998	4432	3828	3828
q4	875	1218	763	763
q5	4048	4376	4342	4342
q6	188	178	145	145
q7	1785	1687	1555	1555
q8	2535	2783	2578	2578
q9	7591	7330	7424	7330
q10	3761	4032	3626	3626
q11	532	447	428	428
q12	517	626	502	502
q13	2780	3165	2330	2330
q14	286	293	282	282
q15	841	827	814	814
q16	710	777	722	722
q17	1223	1518	1407	1407
q18	7197	6869	6681	6681
q19	970	1037	882	882
q20	2085	2184	2012	2012
q21	4056	3492	3398	3398
q22	453	434	357	357
Total cold run time: 51154 ms
Total hot run time: 48617 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 153768 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 85d56d3502b6df0512dc95e2173b421b5c6b05df, data reload: false

query5	4327	633	522	522
query6	328	220	202	202
query7	4210	477	262	262
query8	326	249	236	236
query9	8729	2765	2767	2765
query10	542	396	357	357
query11	7318	5886	5609	5609
query12	185	129	137	129
query13	1289	485	362	362
query14	5665	3946	3625	3625
query14_1	2854	2853	2853	2853
query15	204	191	177	177
query16	973	472	460	460
query17	915	734	616	616
query18	2439	456	379	379
query19	213	215	183	183
query20	134	130	130	130
query21	228	149	129	129
query22	4927	4991	4872	4872
query23	16710	16214	16094	16094
query23_1	15946	15997	15709	15709
query24	7461	1671	1269	1269
query24_1	1313	1388	1384	1384
query25	567	478	430	430
query26	1258	272	149	149
query27	2788	476	287	287
query28	4506	1857	1842	1842
query29	820	563	472	472
query30	312	245	211	211
query31	1357	1287	1211	1211
query32	82	69	79	69
query33	500	347	277	277
query34	942	919	564	564
query35	646	668	594	594
query36	1084	1125	1001	1001
query37	130	94	84	84
query38	2979	2887	2882	2882
query39	903	876	860	860
query39_1	841	831	927	831
query40	235	151	138	138
query41	61	57	58	57
query42	309	301	293	293
query43	247	251	223	223
query44	
query45	198	187	183	183
query46	885	985	603	603
query47	2126	2126	2055	2055
query48	314	310	241	241
query49	627	460	379	379
query50	680	274	210	210
query51	4156	4087	4031	4031
query52	287	290	290	290
query53	291	339	288	288
query54	294	266	265	265
query55	92	83	83	83
query56	313	321	311	311
query57	1302	1343	1323	1323
query58	323	286	278	278
query59	1312	1430	1274	1274
query60	334	353	322	322
query61	146	146	148	146
query62	634	582	537	537
query63	313	283	283	283
query64	4997	1283	998	998
query65	
query66	1464	468	361	361
query67	16307	16391	16272	16272
query68	
query69	387	308	280	280
query70	971	938	966	938
query71	333	308	306	306
query72	2837	2673	2424	2424
query73	548	557	336	336
query74	9987	10014	9819	9819
query75	2857	2773	2493	2493
query76	2297	1044	689	689
query77	385	398	327	327
query78	11114	11289	10678	10678
query79	2685	777	610	610
query80	1751	648	575	575
query81	565	285	242	242
query82	997	159	125	125
query83	337	270	257	257
query84	306	128	106	106
query85	988	552	438	438
query86	411	308	298	298
query87	3143	3124	2980	2980
query88	3590	2682	2658	2658
query89	424	381	351	351
query90	1996	177	181	177
query91	170	159	133	133
query92	76	77	68	68
query93	1151	846	509	509
query94	635	301	286	286
query95	598	338	314	314
query96	633	510	230	230
query97	2468	2481	2423	2423
query98	240	220	217	217
query99	1004	995	915	915
Total cold run time: 236065 ms
Total hot run time: 153768 ms

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.62% (19694/37426)
Line Coverage 36.15% (183699/508103)
Region Coverage 32.28% (141777/439230)
Branch Coverage 33.45% (61924/185122)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100% (0/0) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.54% (26207/36631)
Line Coverage 54.30% (274956/506381)
Region Coverage 51.48% (228202/443278)
Branch Coverage 52.91% (98206/185604)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants